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FOREWORD 


The  Rotary-Wing  Aviation  Research  Unit  of  the  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (ARI)  is  located  at  Fort  Rucker,  Alabama.  The  ARI  Aircrew 
Performance  Team  is  committed  to  enhancing  aviation  training  in  the  Army.  One  means  by 
which  this  can  be  achieved  is  through  the  optimization  of  simulation-augmented  Initial  Entry 
Rotary-Wing  (IERW)  training  programs.  The  advent  of  PC-based  simulators  and  other  training 
devices  which  are  more  reliable,  simpler,  and  much  less  expensive  than  their  predecessors,  could 
potentially  revolutionize  the  expansion  of  synthetic  flight  for  all  levels  of  training.  A  research 
program  in  this  area  was  deemed  necessary  because  few  reliable  benchmarks  exist  as  to  how  this 
new  technology  can  best  be  employed  in  an  IERW  environment. 

Currently,  the  use  of  simulation  in  IERW  training  is  limited  to  the  Instrument  Phase, 
which  begins  after  the  student  pilot  has  mastered  contact  flight  (i.e.,  visual  flight  rules).  The 
simulators  presently  in  use  are  complex,  expensive,  and  based  upon  late  1960s  technology.  The 
research  project  comprising  this  report  concerned  itself  with  the  comparison  of  two  simulator 
technologies:  a  low-cost,  PC -based  simulator,  represented  by  the  Frasca  342  Primary  Skills 
Trainer,  and  the  currently  operational  2B24  Synthetic  Flight  Training  System.  Student  pilots 
were  assigned  to  one  simulator  or  the  other  on  a  random  basis,  and  completed  the  Instrument 
Phase  of  training.  Results  indicated  that  training  outcomes  were  equally  successful,  regardless 
of  the  simulator.  The  major  difference  was  the  much  lower  operational  cost  of  the  Frasca  342. 
The  results  of  the  research  were  briefed  to  the  Commander,  Aviation  Training  Brigade,  on  22 
August  2001.  A  briefing  of  the  preliminary  findings  was  presented  at  the  U.S.  Army  Aviation 
and  Missile  Command  Aviation  Science  and  Technology  Review,  on  27  June  2001. 


TA  M.  SIMUTIS 
Pfechnical  Director 
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ASSESSING  THE  EFFECTIVENESS  OF  A  LOW-COST  SIMULATOR  FOR  INSTRUMENT 
TRAINING  FOR  THE  TH-67  HELICOPTER 


EXECUTIVE  SUMMARY  _ 

Research  Requirement: 

The  U.S.  Army  uses  a  simulator,  the  2B24  Synthetic  Flight  Training  System  (SFTS)  for 
the  Instrument  Phase  of  Initial  Entry  Rotary  Wing  (IERW)  training.  The  SFTS  is  a  dedicated 
instrument  trainer  with  no  visual  display  system.  It  is  mounted  on  a  five  degree  of  freedom 
hydraulic  motion  base.  Its  technology  dates  from  the  late  1960s:  furthermore,  it  represents  the 
UH-1,  which  has  been  replaced  by  the  TH-67  primary  training  helicopter.  The  U.S.  Army 
Aviation  Center  (USAAVNC)  is  concerned  with  the  age,  complexity,  and  operating  costs  of  the 
SFTS,  at  a  time  when  simpler,  more  affordable  and  versatile,  PC-based  simulators  are  available. 
An  example  of  such  a  simulator  is  the  Frasca  342  Primary  Skills  Trainer  (PST)  proof-of-concept 
simulator.  The  PST  has  a  cockpit  modeled  after  the  TH-67,  and  is  equipped  with  a  low-cost 
visual  display  system.  It  was  loaned  to  AR1  by  the  manufacturer  for  purposes  of  conducting 
usability  and  transfer  of  training  research.  USAAVNC’s  quest  for  a  new-technology  simulator 
provided  an  excellent  opportunity  to  conduct  a  training  effectiveness  assessment. 

Procedure: 

The  training  effectiveness  research  compared  two  simulator  technologies  in  a  routine 
training  environment.  Thirty-eight  IERW  flight  students  participated.  They  were  assigned 
randomly  to  either  the  PST  (experimental  group)  or  the  SFTS  (control  group).  A  simple  two- 
group  design  was  used.  The  only  difference  was  the  simulator  in  which  the  participants  received 
the  30  hr  of  instruction.  After  completion  of  the  simulator  phase,  both  groups  reported  to  the 
flightline  to  complete  the  last  20  hr  of  Instrument  Phase  in  the  TH-67.  Five  instructor  pilots  (IPs) 
familiar  with  the  PST  trained  both  experimental  and  control  group  students.  Three 
Standardization  IPs  administered  the  evaluation  checkrides  in  simulator  and  aircraft. 

Findings: 

Few  significant  differences  in  measures  of  performance  (checkride  scores;  hours  to 
proficiency;  usability  ratings  of  the  simulators)  were  evident.  No  student  pilots  in  either 
condition  were  set  back  to  later  classes  or  eliminated  from  training.  On  the  postexperimental 
questionnaire,  control  group  participants  were  more  likely  than  their  experimental  group 
counterparts  to  indicate  that  some  things  learned  in  the  simulator  had  hindered  their  performance 
in  the  aircraft.  The  PST  was  rated  as  significantly  inferior  in  trim  control  to  the  SFTS,  but 
significantly  better  for  instrument  take  offs  and  for  instrument  approaches.  Most  other 
differences  were  nonsignificant,  though  overall  it  appeared  that  the  PST  had  more  appeal  to 
students  than  did  the  SFTS.  The  most  frequent  spontaneous  complaint  from  PST  students 
concerned  poor  trim  control;  for  the  SFTS,  it  was  dissimilarity  of  its  cockpit  from  that  of  the 
TH-67. 
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Utilization  of  Findings: 


The  research  demonstrated  that  EERW  students  could  learn  Instrument  Phase  flight  skills 
in  a  simpler,  more  economical  simulator.  The  complexity  of  the  hydraulically-actuated  flight 
controls  and  the  motion  cueing  system  of  the  SFTS  appear  unnecessary  for  successful  instrument 
training.  Measures  of  performance  and  evaluative  input  from  student  pilots  and  IPs  pinpointed 
PST  technical  issues,  mostly  related  to  software,  that  would  require  resolution  before  this  type  of 
simulator  could  be  acquired  by  the  Army.  In  brief,  the  software  aerodynamic  model  and 
instrumentation  would  have  to  represent  the  TH-67,  both  physically  and  functionally.  Finally, 
the  presence  of  a  visual  display  system  on  the  PST  suggests  that  an  improved  version  of  this  type 
of  simulator  could  support  training  beyond  the  Instrument  Phase.  A  future  research  project, 
employing  such  a  simulator,  could  determine  the  range  of  flight  skills,  both  Primary  (visual  flight 
rules)  and  Instrument  that  could  be  trained. 
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Introduction 


Training  Effectiveness  Research 

Cost  and  training  effectiveness  have  been,  and  still  are.  important  issues  for  transfer  of 
training  (TOT)  research.  The  Army  has  long  been  concerned  with  the  training  effectiveness  of 
simulators  that  are  simpler  and  cheaper,  when  compared  to  those  currently  in  operation  (Caro, 
Jolley,  Isley,  &  Wright,  1972;  Prophet  &  Boyd,  1970).  Paul  Caro  (1988)  discusses  research, 
conducted  in  the  early  1970s,  which  compared  the  training  effectiveness  of  a  high  fidelity, 
expensive  cockpit  procedures  trainer,  for  the  OV-1  aircraft  (Device  2-C-9),  with  a  simpler 
plywood  mock-up  of  the  same  aircraft.  The  mock-up  had  been  built  by  carpenters  at  a  cost  of 
$30.00.  Although  in  many  ways  dissimilar  to  the  aircraft,  the  training  device  had  functional 
fidelity,  in  that  it  could  provide  the  same  critical  cues  for  practicing  cockpit  procedural  tasks,  as 
did  its  high-realism  counterpart.  Caro  defines  both  of  these  devices  as  simulators,  as  opposed  to 
generic  trainers.  His  criterion  for  the  definition  is  that  they  were  designed  to  present  precisely 
the  cues  and  response  opportunities  necessary  for  performing  procedural  tasks  in  the  aircraft.  In 
other  words,  the  spatial  and  functional  relationships  of  the  simulated  levers,  dials,  and  switches 
were  the  same  in  the  low-cost  simulator  as  in  the  OV-L  Transfer  of  skills  to  the  aircraft  should 
be  the  same,  regardless  of  the  cost  of  the  simulator.  Prophet  and  Boyd  (1970)  compared  three 
groups  of  Army  aviators,  all  of  whom  were  transitioning  to  the  OV-1  tactical  reconnaissance 
aircraft.  One  group  trained  in  the  high-fidelity  procedures  trainer,  the  second  in  the  plywood 
mock-up,  and  the  third  trained  exclusively  in  the  aircraft.  Each  group  received  five  trials  in  the 
device,  and  subsequently  five  in  the  aircraft.  On  the  first  trial  in  the  aircraft,  both  training  device 
groups  performed  equally  well,  and  only  slightly  worse  than  the  aircraft-only  baseline  group. 
Prophet  and  Boyd  concluded  that  the  transfer  characteristics  of  both  trainers  were  essentially  the 
same,  despite  cost  differences,  and  that  pretraining  in  the  mock-up  was  about  as  effective  as 
training  in  the  aircraft  alone.  Both  this  study,  and  Caro,  et  al.’s  evaluation  of  a  plywood  mockup 
of  the  U-21  fixed  wing  aircraft  cockpit,  were  practical  applications  of  TOT  methodology.  Their 
main  objective  was  to  determine  the  functionality  of  very  low-cost  simulators  that  were  cheaper 
than  contemporary  training  devices  developed  for  the  Army,  and  far  cheaper  than  the  hourly  cost 
of  training  in  the  aircraft  they  represented.  The  research  confirmed  that  greater  training 
efficiency  is  possible,  to  the  extent  that  low-cost  simulators  can  be  developed,  so  long  as  the 
critical  functions  being  trained  are  preserved. 
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The  issue  addressed  by  Prophet  and  Boyd  is  even  more  important  today.  The  advent  of 
powerful,  affordable,  PC-based  simulator  technology  promises  to  replace  the  older  simulators 
with  a  new  generation  requiring  less  support  and  maintenance.  The  main  difference  between  1970 
and  the  present  is  that  the  less  expensive  alternative  no  longer  has  to  be  a  relatively  crude  mock- 
up.  Instrument  training  is  a  case  in  point. 

Instrument  Training  in  Army  Aviation 

IJ.S.  Army  initial  entry  rotary  wing  CIERW1  flight  training.  The  Instrument  Phase  of 
IERW  lasts  a  total  of  eight  weeks  and  consists  of  50  flight  hours.  Of  these,  30  take  place  in  the 
Synthetic  Flight  Training  System  (SFTS);  the  remainder  in  the  aircraft.  The  SFTS  is  a  dedicated 
instrument  trainer  based  on  the  UH-1  helicopter.  It  is  mounted  on  a  five  degree  of  freedom 
motion  platform,  has  a  high  fidelity  UH-1  cockpit,  hydraulically  loaded  controls,  and  a  complete 
Instructor-Operator  Station  (IOS).  Its  technology  dates  from  the  late  1960s,  and  it  has  been 
operationally  employed  at  the  U.S.  Army  Aviation  Center  (USAAVNC)  at  Fort  Rucker,  AL, 
since  the  early  1970s.  It  is  a  dedicated  instrument  flight  training  simulator;  hence,  it  has  no  visual 
display  system.  SFTS  training,  in  the  IERW  program,  consists  of  20  training  days,  each  lasting 
1 .5  hr.  Training  in  the  TH-67  primary  training  helicopter  spans  20  training  days,  with  each 
session  lasting  1  hr.  Student  pilots  who  begin  the  Instrument  Phase  of  IERW  have  successfully 
completed  the  Primary  Phase,  which  consists  of  a  total  of  60  flight  hr  of  contact  (visual)  flight 
training.  This  includes  successful  completion  of  unsupervised  solo  flight,  and  an  end  of  Primary 
Phase  checkride.  Primary  flight  training  takes  place  on  the  flightline  using  the  TH-67;  no 
simulation  is  employed. 

Training  effectiveness  of  the  SFTS.  Caro  (1972)  performed  an  evaluation  of  the  Army  s 
then-new  SFTS.  He  reported  that  introduction  of  the  simulator  reduced  instrument  training  time 
in  the  aircraft  by  approximately  90%.  Prior  to  the  introduction  of  the  SFTS  and  the  revamped 
instrument  training  program,  60  hr  aircraft  time  and  26  hr  in  a  modified  Link  1-CA-l  Trainer  were 
required  to  obtain  the  Army  Standard  Instrument  Card.  Introduction  of  the  new  simulator  and 
revised  training  program  reduced  aircraft  training  time  to  6.5  hr,  supplemented  by  approximately 
43  hr  simulator  time.  The  length  of  the  instrument  training  course  was  reduced  from  12  to  8 
weeks.  The  SFTS  remains  operational  today,  which  is  a  tribute  to  its  training  effectiveness.  Cost 
effectiveness  is  another  issue  altogether. 

Research  has  shown  that,  with  modifications,  the  SFTS  could  be  used  as  a  primary,  visual 
flight  trainer.  The  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI)  has 
supported  the  Aviation  Training  Brigade  (ATB)  in  the  execution  of  simulation  transfer  of  training 
(TOT)  research  at  Fort  Rucker,  AL.  Research  involving  a  modified  SFTS  was  performed  by  Jack 
Dohme  in  the  late  1980s  and  early  1990s  (Dohme,  1991;  1995).  Dohme  converted  the  SFTS 
into  a  visual  simulator  by  the  addition  of  image  generators  and  three  69  cm  monitors.  The 
software  flight  model  was  also  modified  to  allow  for  hovering  and  low  speed  flight.  Dohme  was 
able  to  demonstrate  that  pretraining  in  this  visual  simulator  saved  training  time  and  task  iterations 
in  the  training  aircraft,  at  that  time  the  UH-1H.  Implications  of  these  findings  were  that  the  SFTS, 
with  similar  modifications,  could  be  used  for  contact  as  well  as  for  instrument  training.  Although 
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the  experiments  were  successful,  USAAVNC  did  not  introduce  simulation  into  the  primary  flight 
training  syllabus. 

Complexity,  cost  and  obsolescence  of  the  SFTS.  ATB  is  concerned  with  the  complexity, 
as  well  as  the  operational  and  maintenance  costs  of  the  aging  SFTS.  Thirty-two  SFTS  cockpits 
are  located  at  Fort  Rucker.  These  are  used  primarily  for  EERW  Instrument  Phase  training.  Each 
training  simulator  requires  complex  hydraulic  equipment  to  operate  the  motion  cueing  system. 
Computer  equipment,  now  outdated,  is  also  required,  as  well  as  a  climate  control  system  to  cool 
the  electronics  and  hydraulics.  If  parts  break,  they  must  sometimes  be  custom  machined, 
replacements  being  unavailable  from  the  original  source  of  supply.  These  drawbacks 
acknowledged,  its  lack  of  versatility  remains.  The  aerodynamic  software  model  does  not  allow 
for  the  practice  of  low  speed  or  hovering  flight;  some  flight  maneuvers  (e.g.,  termination  to  a 
hover)  cannot  be  performed.  Furthermore,  the  need  for  the  expensive  and  complex  motion 
cueing  system  has  never  been  demonstrated.  Finally,  the  SFTS  represents  an  interim  training 
aircraft  that  has  been  withdrawn  from  service.  The  UH-1  configuration  of  flight  instruments  is 
quite  different  from  that  of  the  TH-67.  This  requires  student  pilots  to  re-adapt  their  instrument 
scan  patterns  to  the  aircraft  when  they  go  to  the  flightline.  All  of  these  concerns  pose  a  cogent 
argument  for  replacing  the  SFTS  at  a  time  when  cheaper,  more  versatile,  and  more  supportable 
simulators  are  available. 

The  Frasca  342  Primary  Skills  Trainer  (PST)  Low-Cost  Flight  Simulator 

The  PC-based  Frasca  342  PST,  on  loan  to  ARI  by  the  manufacturer,  (Frasca  International, 
Inc.,  Urbana,  IL)  is  a  proof-of-concept  testbed  for  a  simulator  that  is  both  more  cost  effective  and 
more  versatile  than  the  SFTS.  The  PST  is  configured  to  simulate  the  characteristics  of  the  TH-67 
primary  training  aircraft.  The  cockpit  is  modeled  after  that  of  the  aircraft  with  dual  flight  controls, 
as  well  as  actual  aircraft  flight  instruments  and  system  indicators.  The  avionics  can  be  used  for 
training  instrument  flight  rules  (IFR)  operations.  The  instrument  panel  does  depart  from  that  of 
the  IFR-configured  TH-67  on  one  important  aspect.  The  PST  has  the  full  complement  of  flight 
instruments  on  the  right  (pilot)  side  only.  Any  operational  instrument  trainer  would  require  a 
Military  Standard,  two-pilot  IFR  cockpit.  For  this  study,  wallpaper  instruments  were  applied  to 
the  left  (copilot)  side  of  the  panel,  to  give  students  a  perspective  on  where  the  flight  instruments 
are  located  on  that  side  for  the  aircraft. 

The  flight  controls  incorporate  a  passive  spring  control  loading  system  designed  to 
produce  a  control  feel  similar  to  that  of  the  aircraft.  A  force  trim  system  for  cyclic  pitch  and  roll 
control  is  included.  Engine,  electrical,  hydraulic,  and  mechanical  failures  can  be  simulated.  A 
variety  of  environmental  conditions  including  wind  speed,  wind  direction,  turbulence,  visibility, 
cloud  ceiling,  and  day/dusk/night  illumination  can  also  be  simulated. 

The  simulator  provides  helicopter  sound  cues,  including  engine,  transmission,  main  rotor, 
and  wind  noise,  plus  warning  tones.  The  out-the-window  view  is  projected  onto  three  screens  at 
the  front  of  the  cockpit.  The  visual  field  of  view  subtended  by  the  front  screen  is  99  degrees 
horizontal  by  25  degrees  vertical,  with  640  x  480  pixels.  The  computer-generated  imagery  in  the 
database  includes:  an  airfield  and  runway,  with  approach  and  runway  lights,  taxi  lanes,  helipad, 
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hangars,  and  control  tower.  A  stage  field  with  a  tower,  helipad,  two  parking  pads,  and  a 
functioning  windsock  are  also  included,  as  are  navigational  beacons  and  ground  stations.  All 
locations  in  the  visual  database  are  accurately  modeled  and  internally  consistent  in  terms  of 
latitude,  longitude,  and  magnetic  compass  orientation.  Four  Pentium  75  computers  control  the 
host  computer  and  visual  display  system. 

Integrated  into  the  simulator  system  is  an  IOS,  consisting  of  a  generic  desktop  PC  (486 
DX  66)  running  an  interactive,  Windows  3 .  l™-based  program.  From  this  station  the  instructor, 
or  a  console  operator,  can  control  environmental  conditions,  system  failures,  and  aircraft  location. 
The  instructor,  or  operator,  can  selectively  freeze  any  or  all  flight  controls  to  allow  training  of 
specific  perceptual-motor  cues  and  skills.  He  or  she  can  track  the  aircraft  s  location  in  three 
dimensions,  monitor  in  real  time  a  wide  range  of  flight  data  (including  aircraft  location  in  six 
dimensions:  x,  y,  z,  pitch,  roll,  yaw),  select  and  record  key  snapshots  of  these  data,  and  print  the 
recorded  file  of  these  snapshots.  Table  1 ,  below,  compares  the  major  components  of  the  PST  to 
those  of  the  SFTS. 

Table  1 


Frasca  Primary  Skills  Trainer  VS.  Synthetic  Flight  Training  System 


Primary  Skills  Trainer  (PST) 

Synthetic  Flight  Training  System  (SFTS) 

1990s  Technology 

1970s  Technology. 

No  Motion  System. 

Motion  System  (Five  degrees  of  freedom). 

Full  Aerodynamic  Flight  Model. 

Flight  Model  at  Speeds  >  40  KT. 

Spring-loaded  Controls. 

Hydraulically-loaded  Controls. 

PC-based  (Pentium  75;  80486). 

Non-PC-based. 

Visual  Display  System. 

No  Visual  Display  System. 

No  Console  Operator  Required. 

Console  Operator  Required. 

TH-67  Cockpit. 

UH-1H  Cockpit. 

Overview 


Method 


This  training  effectiveness  assessment  compared  the  performance  of  two  training  devices, 
in  a  routine,  operational  training  environment.  Before  such  a  study  could  be  undertaken,  the 
Frasca  PST  had  to  be  evaluated  by  U.S.  Army  military,  civilian,  and  contractor  instructor  pilots 
(IPs)  to  determine  if  basic  instrument  (BI)  and  advanced  instrument  (AI)  flight  maneuver  tasks 
could  be  adequately  trained  in  the  simulator.  Certification  by  ATB  is  required  for  any  such  use  of 
a  simulator  or  training  device.  This  was  important  because,  unlike  other  research  in  which  student 
pilots  are  pretrained  in  a  simulator,  in  this  study  the  PST  would  be  employed  as  an  operational 
instrument  trainer.  The  IPs  who  would  be  conducting  training  in  the  simulator  would  also  need  to 
be  familiarized  with  the  device.  They  were  asked,  in  the  course  of  their  training,  to  note  any 
discrepancies  between  the  PST’s  instruments  and  handling,  and  those  of  the  TH-67.  Student 
pilots  were  selected  from  successive  classes  and  assigned  either  to  the  PST  or  to  a  comparison 
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SFTS  simulator.  The  same  U.S.  Army  Aviation  Program  of  Instruction  (POI)  that  is  used  for  all 
IERW  instrument  training  was  employed. 

Participants  and  Design 

Participants  were  38  EERW  flight  students  from  even-numbered  classes  (2000-14  through 
2001-08),  who  were  starting  the  Instrument  Phase  of  IERW.  Data  collection  began  27  July  2000 
and  ended  25  June  2001 .  Mean  age  was  26. 1 1  years,  (SD  =  3. 14);  mean  score  on  the  Flight 
Aptitude  Selection  Test  (FAST)  was  128.52  (SD  =  12.56).  Of  the  38  participants,  30  were  male, 
34  described  themselves  as  Caucasian,  four  as  Hispanic.  The  sample  consisted  of  1  First 
Lieutenant,  18  Second  Lieutenants,  and  19  Warrant  Officer  Is.  Only  those  student  pilots  with 
midrange  grades  (84-89)  in  Primary  Phase  training  were  selected.  Those  who  had  been  set  back 
from  earlier  classes,  or  awarded  additional  hours  due  to  performance  deficiencies  were  not  eligible 
for  selection,  nor  were  those  with  primary  flight  grades  90  and  above.  Students  with  flight 
experience,  prior  to  EERW,  were  also  excluded.  A  two-group  TOT  paradigm  was  used.  The  only 
difference  between  the  two  groups  was  the  simulator  in  which  they  received  the  first  30  hr  of 
instruction.  The  experimental  group  trained  in  the  PST,  the  control  group  in  the  SFTS.  Both 
groups  then  went  to  the  flightline  for  20  hr  of  training  in  the  TH-67  helicopter.  Two  participants 
(“stick  buddies”)  were  assigned  to  each  group.  They  were  matched,  based  on  their  performance 
in  primary  training.  The  sample  was  balanced  by  gender,  so  that  when  a  female  student  was 
assigned  to  the  experimental  group,  another  was  assigned  to  the  control  group.  One  important 
consideration  in  selection  of  participants  was  weight.  This  was  dictated  by  performance 
limitations  of  the  TH-67,  especially  during  the  hot,  humid  summer  months.  For  this  reason,  no 
student  pilots  weighing  in  excess  of  86  kg  were  selected.  The  aircraft  normally  carries  two 
student  pilots  and  one  IP. 

Procedure 


Five  civilian  contract  IPs  from  two  training  flights  at  Fort  Rucker,  who  volunteered  to 
participate  in  the  study,  were  trained  to  use  the  Frasca  PST.  All  were  Caucasian  males.  One 
withdrew  from  the  project  after  training  two  students  in  the  SFTS.  For  all  five,  mean  age  was  51 
years  (SD  -  8.49).  The  youngest  was  39;  the  oldest,  59.  Total  flight  hours,  in  a  variety  of 
aircraft  types,  ranged  from  4,550  hr  to  15,000  hr  (M  =  10,487.50,  SD  =  4,347.87).  All  had  had 
extensive  experience  training  students  in  the  SFTS.  For  the  four  who  completed  the 
postexperimental  questionnaire,  experience  as  IPs  ranged  from  14  to  31  yr  (M  =  25.0;  SD  = 

8.04).  Experience  as  Instrument  Phase  IPs  ranged  from  1.5  to  24  yr  (M  =  11-13,  SDj=  9.80). 
They  also  had  provided  evaluative  input  on  the  suitability  of  the  P  ST  for  training  the  standard 
flight  maneuvers  for  the  Instrument  Phase  of  IERW.  Each  IP  was  responsible  for  training  a  pair 
of  stick  buddies.  They  would  alternate  between  PST  and  SFTS  for  each  successive  class  trained, 
in  order  to  maintain  balance  in  the  research  design.  Each  pair  of  stick  buddies  would  have  the 
same  training  IP  throughout  the  Instrument  Phase.  Three  U.S.  Army  standards  and  evaluation  IPs 
(SEPs)  administered  the  Basic  Instrument  (BI)  checkride  in  the  simulator  after  12  hr  of  training 
had  been  completed,  and  the  final  advanced  instrument  (AI)  checkride  in  the  aircraft,  after 
completion  of  the  50  hr  Instrument  Phase,  for  both  treatment  conditions.  Efforts  were  undertaken 
for  the  same  SIP  to  administer  both  checkrides  (BI  in  the  simulator,  AI  in  the  aircraft)  for  the  pair 
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of  participants  in  the  experimental  or  control  condition,  though  this  was  not  always  possible.  For 
these  three  SIPs,  all  Caucasian  males,  mean  age  was  48.67  years  (SD  =  7.77).  The  youngest  was 
40,  the  oldest,  55.  Total  flight  hours,  in  a  variety  of  aircraft  types,  ranged  from  3,400  hr  to 
14*000  hr  (M  =  10,433.33;  SD  =  6,091 .25).  SIPs,  like  the  IPs,  alternated  between  experimental 
and  control  participants,  for  consecutive  classes,  whenever  possible. 

During  the  BI  phase  of  instrument  training,  the  student  learns  the  fundamentals  of  flying 
the  aircraft,  without  visual  reference  cues,  in  controlled  airspace.  For  example,  the  student  must 
learn  to  make  standard  rate  (3°/sec)  turns,  timed  turns,  climbs,  descents,  and  to  accelerate  and 
decelerate,  all  within  the  standards  of  IFR.  The  AI  phase  teaches  the  fundamentals  of  radio 
navigation,  which  includes  holding  and  approach  procedures,  using  a  variety  of  radio  navigation 
aids,  including  non-directional  beacons  (NDBs),  very  high  frequency  omni  directional  radio  range 
(YOR)  transmitters,  runway  approach  localizers  (LOC),  and  instrument  landing  system 
(ILS)  aids. 

Two  retired  Army  aviators,  one  a  civilian  Department  of  the  Army  computer  specialist,  the 
other  a  contractor,  alternated  as  IOS  operators.  One  had  previously  been  rated  in  OH-58  A/C 
helicopters;  the  other,  in  various  models  of  the  AH-1  helicopter.  They  were  supplemented  by  an 
active  duty  Army  aviator,  a  Chief  Warrant  Officer  2,  rated  in  the  AH-64A  helicopter,  who  was  on 
casual  assignment. 

Measures  of  Performance 

Flight  grades.  During  the  Instrument  Phase  of  IERW,  student  progress  is  evaluated  on 
a  daily  basis,  from  Training  Day  61  until  the  final  checkride  on  Training  Day  100.  The 
cumulative  hours  are  noted,  and  a  letter  grade  is  given  for  each  maneuver  task  performed. 

A  grade  of  B  indicates  performance  of  the  task  to  standard.  Any  grade  less  than  a  B  indicates  that 
performance  is  substandard  for  a  particular  maneuver.  Performance  to  standard  indicates 
successful  performance  of  the  task,  with  reference  to  formal  criteria  as  published  in  the  Instrument 
Phase  Flight  Training  Guide  (USAAVNC,  2001).  All  flight  maneuver  tasks  are  defined  in  terms 
of  relevant  parameters  (e.g.  airspeed,  altitude,  heading)  and  the  deviation  allowed  (e  g.,  plus  or 
minus  10°).  A  student  not  performing  with  the  specified  parameters  for  a  given  task  has  not  met 
the  standard.  Performance  to  standard  is  not  expected  of  a  student  the  first  time  that  he  or  she 
attempts  it;  in  this  instance,  a  grade  of  NM  (standard  not  met)  is  appropriate.  The  grade  sheets 
(USAAVNC  Form  463)  used  for  instrument  training  clearly  delineate  the  hourly  point  at  which 
performance  to  standard  (P3  level)  is  expected,  for  each  maneuver  task.  The  culmination  of  BI 
and  AI  is  a  checkride,  administered  by  an  evaluation  SIP  who  did  not  provide  the  daily  training. 
Checkride  performance  is  indicated  by  numerical  scores,  not  grades  (although  these  are 
traditionally  referred  to  as  grades).  The  training  IP  attempts  to  estimate  the  student’s 
performance  on  the  checkride  by  providing  a  “put-up  score  independent  of  the  final  checkride 
score.  The  SIP  does  not  know  the  put-up  score  while  administering  the  checkride.  Put-up  and 
checkride  scores  are  then  averaged  into  an  evaluation  grade.  Both  put-up  and  checkride  grades 
have  their  advantages;  the  training  IP  is  familiar  with  the  student  s  performance  over  a  long  period 
of  trials,  whereas  the  SEP,  who  has  not  observed  the  student  over  this  period,  observes  a 
“snapshot”  of  his  or  her  performance. 
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Student  proficiency  records.  Each  IP  was  asked  to  complete  a  proficiency  record  (see 
Appendix  A)  for  each  student  participant,  noting  the  hourly  point  at  which  he  or  she  demonstrated 
performance  to  standard  (proficiency)  in  the  simulator  for  BI  tasks,  and  later  in  the  aircraft,  for  AI 
tasks.  Fourteen  BI  maneuvers  and  twelve  AI  maneuvers  were  included  on  the  form.  This  form 
was  supplemental  to  the  much  more  comprehensive  Form  463,  and  was  employed  in  the  study 
because  the  data  could  be  tabulated  much  more  easily  and  rapidly. 

Postexperimental  questionnaire.  For  purposes  of  assessing  training  usability,  supplemental 
questionnaires  were  administered  to  PST  and  SFTS  participants  (Appendix  B).  A  modified  form 
of  this  same  questionnaire  was  administered  to  the  four  IPs  who  participated  in  the  daily  training 
of  students  in  the  PST  and  SFTS.  These  questionnaires  asked  the  student  to  evaluate  the  training 
effectiveness  of  the  simulator,  for  specific  maneuver  tasks  that  had  been  performed.  Students 
were  also  encouraged  to  make  spontaneous  comments  about  the  simulator  and  training 
experience. 

Use  of  desktop  flight  simulators.  Another  supplementary  questionnaire  (Appendix  C), 
given  to  each  student  at  the  outset  of  instrument  training,  was  designed  to  elicit  information  as  to 
whether  or  not  he  or  she  had  used  a  desktop  PC-based  flight  simulator  (e.g.,  Microsoft  Flight 
Simulator  2000),  and  if  yes,  for  how  many  hours  in  the  past  month.  The  intent  of  this  one  page 
questionnaire  was  to  determine  if  prior  use  of  popular  aviation-oriented  software  and  gameware 
conferred  any  advantage  in  learning  the  IERW  instrument  maneuvers. 

Hypotheses 

The  PST  is  closer  in  configuration  to  the  TH-67  than  is  the  SFTS.  Consequently,  it  would 
be  reasonable  to  expect  that  student  pilots  would  reach  proficiency  in  the  aircraft  sooner  if  trained 
in  the  PST  rather  than  the  SFTS.  This  is  because  instrument  scan  patterns  learned  in  the 
PST  should  transfer  to  the  TH-67  more  readily;  whereas  those  learned  in  the  SFTS  would  have  to 
be  unlearned,  once  the  student  began  training  on  the  flightline.  However,  it  is  acknowledged  that 
this  advantage  may  be  short-lived,  in  that  20,  1  hr  training  sessions  are  spent  in  the  aircraft, 
providing  sufficient  time  for  re-adaptation. 


Results 

Participants 

Two  participants,  both  in  the  SFTS  (control)  group,  were  eliminated  from  the  study  for 
medical  reasons  unrelated  to  the  research.  Eliminations  in  both  cases  occurred  too  late  for  an 
alternate  to  serve  as  a  replacement.  In  both  instances,  the  student  pilots  had  completed  the 
simulator  phase  of  training,  and  their  classes  had  gone  to  the  flightline  for  training  in  the  TH-67. 
The  final  sample  size  of  38  participants  comprised  20  in  the  PST  group,  and  18  in  the  SFTS 
group.  Mean  age  of  the  PST  participants  was  26.85  years  (SD  =  3.23)  vs.  25.28  years  for  the 
SFTS  (SD  =  2.87);  this  difference  was  nonsignificant.  Mean  FAST  score  for  the  PST  group  was 
121.20  (SD  =  15.40),  vs.  130.04  (SD  =  14.62)  for  the  SFTS  group.  This  difference  approached 
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significance  (t  =  -1 .81,  df  =  36,  p  <  .08).  Since  actual  FAST  scores  were  not  easily  obtainable 
from  records,  the  investigators  had  to  rely  on  self-reports. 

Technical  Problems 

At  the  start  of  the  evaluation  of  the  P  ST,  IPs  noticed  some  technical  issues. 

Those  that  were  considered  serious  enough  to  prevent  the  research  from  being  conducted  (erratic 
altimeter  readings,  inoperative  attitude  indicator,  inoperative  and  occasionally  erratic  turn 
indicator)  were  resolved.  After  consulting  with  the  EPs,  and  ATB  Flight  Standards,  the  research 
team  decided  to  proceed  with  the  project,  in  spite  of  several  problems  and  anomalies  that  were  as 
yet  unresolved.  These  were:  A  tendency  to  gain,  rather  than  lose  altitude  in  a  turn,  calibration  of 
the  turn  needle  and  attitude  indicator  that  were  non-standard,  an  inoperative  trim  ball,  inconsistent 
readings  from  the  magnetic  compass,  and  anomalous  readings  on  the  torque  indicator.  Initially,  it 
was  feared  that  these  unresolved  problems  would  preclude  the  effective  instruction  of  BI  skills; 
however,  IPs  learned  to  work  around  these  deficiencies,  so  that  both  BI  and  AI  skills  could  be 
taught.  These  issues  were  all  due  to  problems  with  the  software  flight  model. 

Measures  of  Performance 

Checkride  scores.  Due  to  heterogeneity  of  variance  considerations  for  a  few  of  these 
measures  in  the  present  study,  and  for  the  sake  of  consistency,  the  Mann-Whitney  U-test  was 
employed  for  all  measures  of  performance.  The  Mann-Whitney  U  test  has  been  described  as  one 
of  the  most  robust  of  the  nonparametrics  (Hays,  1973).  In  cases  where  the  larger  of  the  two 
samples  contains  20  or  more  observations,  the  statistic  approximates  the  normal  distribution 
and  can  be  interpreted  as  the  normal  deviate  z.  BI  and  AI  put-up  and  checkride  scores  were 
compared  for  the  two  groups  of  students.  Only  AI  put-up  scores  showed  any  significant 
difference  between  conditions  (z 7  1 .96,  p  <  .05),  indicating  that  IPs  anticipated  better 
performance  from  PST  than  from  SFTS  participants.  Put-up  and  checkride  grades  appear  in 
Table  2.  No  student  pilots  in  either  the  PST  or  SFTS  groups  were  set  back  to  later  classes  or 
eliminated  from  flight  training;  all  completed  the  Instrument  Phase  successfully  and  graduated 
with  their  classes. 

Table  2 


Mean  Instrument  Phase  Put-up  and  Checkride  Scores  (SDs  in  Parentheses) 


Condition 

N 

Basic 

Instruments 
(BI)  Put-up 

Basic 

Instruments 
(BI)  Checkride 

Advanced 
Instruments  (AI) 
Put-up 

Advanced 
Instruments 
(AI)' Checkride 

Experimental 

(PST) 

20 

89.05  (3.33) 

87.50  (7.42) 

89.00(4.19) 

85.35  (7.28) 

_ \  / - 

Control 

(SFTS) 

18 

89.33  (2.91) 

88.83  (4.09) 

86.11  (4.90) 

85.94  (6.80) 
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Hours  to  proficiency.  The  supplementary  proficiency  record  was  designed  to  capture,  in 
the  IP’s  judgment,  the  hourly  point  at  which  the  student  pilot  performed  14  BI  and  12  AI 
maneuver  tasks  to  standard  (a  grade  of  B).  As  in  the  case  of  traditional  flight  grades,  the 
comparisons  of  proficiency  times  showed  few  significant  differences  between  groups.  Means, 
SDs,  and  significance  levels  for  BI  tasks  appear  in  Table  3;  those  for  AI  tasks  appear  in  Table  4. 
For  the  BI  tasks  listed  in  Table  3,  standard  rate  turns,  in  emergency  panel  mode,  showed  a 
significant  difference  favoring  the  SFTS.  Climbs  and  descents,  in  emergency  panel  mode,  also 
showed  a  difference  favoring  the  SFTS,  but  this  did  not  reach  conventional  levels  of  significance. 
Partial  or  emergency  panel  signifies  a  situation  in  which  certain  key  instruments  (e.g.,  attitude 
indicator)  fail,  and  the  pilot  must  use  the  remaining  instruments  to  control  the  aircraft.  U-tests 
performed  on  the  AI  maneuver  tasks  revealed  no  significant  between-group  differences  in  hours 
to  proficiency,  though  for  all  12  maneuvers,  PST  students  performed  to  standard  in  .2  to  3.0  hr 
less  time  than  did  their  SFTS  counterparts,  with  an  overall  mean  difference  of  1 .54  hr.  Although 
these  differences  were  not  significant,  the  time  savings  may  be  meaningful,  since  much  of  AI  time 
is  aircraft  time,  which  is  costlier  than  time  in  the  simulator.  Still,  it  must  be  concluded  that 
students  trained  in  the  PST  had  no  advantage  over  those  trained  in  the  SFTS,  not  supporting  the 
hypothesis  that  the  former  would  attain  proficiency  in  fewer  hours.  A  caveat  is  in  order:  The  two 
significant  differences  in  Table  3  could  be  so  simply  by  chance,  judging  from  their  small  number. 
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Table  3 


Mean  Hours  to  Proficiency  for  Basic  Instruments  (BI)  Flight  Maneuvers 


Maneuver 

Condition 

Mean 

SD 

_ 

Accelerate/Decelerate 

PST 

9.43 

1.57 

.82 

SFTS 

9.29 

1.98 

Climbs/  Descents 

PST 

5.90 

2.06 

.19 

SFTS 

7.04 

2.43 

Climbs/  Descents,  Emergency 
Panel 

PST 

10.04 

1.28 

.06 

SFTS 

9.19 

1.61 

Climbing/Descending  Turn 

PST 

9.45 

1.61 

.73 

SFTS 

9.62 

2.19 

Climbing/Descending  Turn, 
Emergency  Panel 

PST 

10.99 

1.25 

.10 

SFTS 

9.93 

2.27 

Simulated  Engine  Failure 

PST 

8.23 

1.59 

.13 

SFTS 

9.09 

1.82 

Standard  Rate  Turn 

PST 

5.78 

2.17 

.33 

SFTS 

5.09 

1.69 

Standard  Rate  Turn, 

Emergency  Panel 

PST 

9.72 

1.50 

.005 

SFTS 

8.24 

1.52 

Steep  Turn 

PST 

7.30 

2.20 

.36 

SFTS 

7.93 

2.30 

Straight  and  Level  Flight 

PST 

4.38 

1.13 

.09 

SFTS 

3.95 

1.11 

Straight  and  Level  Flight, 
Emergency  Panel 

PST 

8.43 

1.99 

.19 

SFTS 

7.71 

1.13 

Timed  Turn 

PST 

6.55 

2.03 

.93 

SFTS 

6.68 

2.17 

Unusual  Attitude  Recovery 

PST 

7.18 

1.39 

.17 

SFTS 

8.07 

1.81 

Unusual  Attitude  Recovery, 
Emergency  Panel 

PST 

9.68 

1.14 

.15 

SFTS 

9.10 

1.49 
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Table  4 

Mean  Hours  to  Proficiency  for  Advanced  Instruments  (AI)  Flight  Maneuvers 


Maneuver 

Condition 

Mean 

SD 

E< 

En  Route  Navigation 

PST 

33.57 

3.75 

.18 

SFTS 

34.98 

4.89 

Instrument  Landing  System 
(ILS)  Approach 

PST 

38.38 

4.43 

.94 

SFTS 

38.57 

6.27 

Instrument  Takeoff 

PST 

35.42 

3.64 

.28 

SFTS 

36.53 

3.79 

Localizer  Approach 

PST 

38.60 

4.69 

.14 

SFTS 

40.78 

4.63 

Localizer  Holding 

PST 

39.30 

5.55 

.57 

SFTS 

40.26 

6.12 

Lost  Communication 

PST 

36.54 

5.97 

.29 

SFTS 

38.77 

4.28  1 

Non  Directional  Beacon  (NDB) 
Approach 

PST 

37.09 

5.69 

.31 

SFTS 

38.94 

6.34 

Non  Directional  Beacon  (NDB) 
Holding 

PST 

37.08 

3.44  1 

.70 

SFTS 

38.44 

5.83 

Precision  Approach 

PST 

36.49 

5.37 

.35 

SFTS 

38.06 

4.89 

Radio  Communication 

PST 

35.67 

3.89 

.13 

SFTS 

38.72 

6.04 

VHF  Omni  Directional  Radio 
(VOR)  Missed  Approach 

PST 

34.92 

3.34 

.33 

SFTS 

36.62 

4.50 

VOR  Holding 

PST 

37.85 

4.20 

.73 

SFTS 

38.74 

5.56 

Student  postexperimental  questionnaire  responses.  Recall  that  student  pilots  were 
administered  a  questionnaire  in  order  to  assess  their  perceptions  of  the  training  effectiveness  of 
their  respective  simulators.  They  were  also  given  the  opportunity  to  provide  additional  evaluative 
input  via  an  open-ended  question  at  the  end  of  the  questionnaire.  The  reader  should  note  that  the 
postexperimental  questionnaire  does  not  include  all  of  the  same  tasks  that  were  sampled  as 
measures  of  proficiency.  AI  tasks  in  particular,  are  subsumed  under  broader  categories.  This  was 
done  primarily  to  keep  the  number  of  questionnaire  items  at  a  manageable  level,  to  insure  a  high 
rate  of  compliance.  Means  and  standard  deviations  for  each  item  rated  appear  in  Table  5.  All 
items  are  presented  in  the  order  in  which  they  appeared  on  the  questionnaire.  One  item, 
pertaining  to  the  adequacy  of  the  visual  display  system  on  the  PST,  was  omitted  because  its 
ambiguity  led  to  widespread  misinterpretation.  An  examination  of  these  responses  shows  that,  in 
general,  students  rated  both  the  PST  and  SFTS  as  training-effective,  though  the  PST  tended  to 
enjoy  more  favorable  ratings  (though  not  significantly  different  in  most  instances).  The  most 
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noteworthy  finding,  for  purposes  of  this  research,  is  the  significant  tendency  of  SFTS  participants 
to  indicate  that  training  in  the  device  hindered  their  performance  in  the  TH-67  (item  7;  see 
Appendix  B),  as  opposed  to  their  PST  counterparts. 

Table  5 

Mean  Student  Questionnaire  Ratings  (6-pt.  Scale)1  by  Item  Content 


Description 

PST 

SFTS  ' 

e2< 

Mean 

SD 

Mean 

SD 

Attitudes  Toward  Simulation 

Simulation  is  a  useful  tool. 

5.45 

.69 

5.33 

.67 

.56 

Simulation  saves  time  in  the  aircraft. 

5.30 

.87 

5.17 

.92 

.65 

Simulator  can’t  be  built  that  handles  like  aircraft. 

4.05 

1.00 

3.78 

1.00 

.50 

Time  saved  in  aircraft  from  simulation  is  negligible. 

2.95 

1.01 

2.89 

1.02 

.84 

Tasks  learned  in  a  simple  simulator  will  transfer  to 
aircraft. 

4.95 

.51 

5.11 

.76 

.42 

Some  things  learned  in  the  simulator  hindered  my 
performance  in  the  aircraft. 

2.55 

1.43 

3.56 

1.38 

.03 

Simulator  Effectiveness  Ratings:  Basic  Instruments 


Straight  and  level  flight. 

5.40 

.68 

4.67 

1.19 

.03 

Timed  turns. 

5.20 

.83 

4.72 

1.18 

.15 

Steep  turns. 

5.00 

1.08 

4.83 

.79 

.30 

Climbs  and  descents. 

5.05 

.76 

4.67 

.97 

.24 

Trim  control. 

2.45 

1.79 

3.61 

1.46 

.02 

Standard  rate  turns. 

5.00 

1.12 

4.89 

.90 

.51 

Climbing  turns. 

4.80 

1.06 

4.89 

.90 

.24 

Acceleration/  Deceleration. 

5.00 

1.03 

4.61 

.85 

.13 

Descending  turns. 

4.85 

.99 

4.83 

.86 

.89 

Simulated  engine  failure  at  altitude. 

4.80 

1.00 

4.65 

1.06 

.68 

Unusual  attitude  recovery. 

5.30 

.87 

5.17 

.79 

.54 

L/UllUlUtV/l  T  i^v.****^.  - - 

Instrument  takeoff. 

4.45 

1.47 

3.38 

1.54 

.03 

Missed  approaches. 

5.15 

1.09 

4.72 

1.02 

.14 

Holds. 

5.45  1 

.76 

4.78 

1.31 

.07 

Instrument  approaches. 

5.55 

.76 

4.72 

1.27 

.02 

1.  A  high  rating  indicates  strong  agreement  or  that  the  simulator  was  very  effective. 

2.  Comparisons  were  via  Mann- Whitney  U  test. 
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4. 


In  addition  to  the  Likert-based  scaled  question  items,  space  was  provided  for  comments 
at  the  end  of  the  questionnaire.  A  total  of  nine  control  group  and  13  experimental  group 
participants  responded  with  open-ended  comments.  Table  6  presents  a  summary  of  the  comments 
by  category.  The  reader  should  be  aware  that  most  participants  who  made  spontaneous 
comments  made  several.  A  glance  at  this  table  shows  that,  among  participants  who  trained  in  the 
PST,  problems  with  trim  control  was  the  foremost  concern.  The  second  most  frequently 
mentioned  deficiency  was  the  difficulty  managing  power,  especially  in  turns.  Collective  pitch 
control  adjustments  were  seen  as  challenging.  Three  participants  remarked  that,  at  the  very  least, 
a  functioning  horizontal  situation  indicator  (HSI)  was  needed  on  the  copilot’s  (left)  side  of  the 
cockpit.  Other  comments  concerned  what  seemed  like  a  fixed-wing  aerodynamic  model,  and  the 
lack  of  a  motion  cueing  system.  In  spite  of  the  perceived  deficiencies,  there  were  six  spontaneous 
laudatory  comments  about  the  Frasca  PST.  These  took  the  form  of  having  enjoyed  the  training 
experience,  and  how  the  Frasca  PST  had  the  potential  of  becoming  an  outstanding  training  device, 
if  the  indicated  problems  were  fixed.  There  were  also  positive  comments  on  the  visual  display 
system,  and  its  potential  as  a  confidence-builder  for  Instrument  Meteorological  Conditions  (IMC) 
final  approaches  and  break-outs.  By  contrast,  most  spontaneous  comments  regarding  the  SFTS 
were  concerned  with  those  differences,  primarily  instrumentation  and  aerodynamic  model,  which 
hindered  transfer  of  instrument  skills  to  the  aircraft.  Most  of  these  concerned  the  necessity  of 
relearning  navigation,  radio,  and  instrument  cross-check  procedures  in  the  TH-67  on  the 
flightline.  One  participant  believed  that  adding  a  visual  display  would  greatly  enhance  the  SFTS’s 
effectiveness.  There  were  three  positive  statements  about  the  training  effectiveness  of  the  SFTS. 

Table  6 

Content  Categories  of  Spontaneous  Comments  of  Participants  on  Postexperimental  Questionnaire 


Simulator 

Content  Category 

Number  of 
Mentions 

PST 

Problems  with  trim  control. 

11 

General  positive  comments  about  simulator  effectiveness 
and  training  experience. 

6 

Visuals  were  helpful. 

5 

Avionics,  instruments  transfer  to  TH-67. 

5 

Power  adjustments  difficult,  especially  in  turns. 

5 

Need  HSI  on  copilot  side  also. 

3 

Simulator  flies  like  fixed-wing  aircraft. 

2 

Simulator  should  have  motion  cueing  system. 

1 

SFTS 

Simulator  too  different  in  control  touch,  instruments,  from 
TH-67. 

8 

Simulator  good  for  teaching  basic  principles;  had  very  little 
trouble  adjusting  to  TH-67. 

3 

Visual  display  would  be  helpful. 

1 
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Instructor  pilot  questionnaire  responses.  The  four  IPs,  who  trained  student  pilots  for  the 
duration  of  the  study,  were  administered  essentially  the  same  postexperimental  questionnaire. 
Minor  semantic  changes  were  made  to  it,  so  that  the  perspective  was  that  of  instructor  rather  than 
student.  The  small  number  of  responses,  plus  the  fact  that  these  were  within-  and  not  between- 
subject  ratings,  renders  statistical  tests  of  differences  impractical.  Nevertheless,  a  side-by-side 
comparison  of  IPs’  and  students’  evaluations  of  the  two  training  devices  may  be  useful.  Table  7 
compares  DP  and  student  pilot  attitudes  toward  simulation  as  a  training  tool,  based  upon  responses 
to  these  five  questions  in  the  post  questionnaire.  Table  8  presents  the  means  and  standard 
deviations  for  IPs’  ratings  of  training  effectiveness,  comparing  the  two  simulators.  The  first  five 
questions,  which  concerned  the  respondents’  attitudes  toward  simulation  in  general,  and  were 
irrelevant  to  comparison  between  the  simulators,  were  only  asked  once.  Table  9  is  a  supplement 
to  Table  8,  comparing  directly  the  ratings  given  by  DPs  to  those  given  by  student  pilots  (SPs).  An 
examination  of  Table  9  shows  that  student  pilots  tended  to  be  more  favorable  toward  the  Frasca 
PST  and  less  favorable  toward  the  SFTS,  than  did  instructors.  It  is  also  quite  evident  that  the 
main  perceived  deficiency  of  the  PST  involved  trim  control. 

Table  7 

Comparison  of  Instructor  Pilot  (IP)  vs  Student  Pilot  (SP)  Attitudes  toward  Simulation 
(6-pt.  Scale)1 


Description 

IP 

SP 

Mean 

SD 

Mean 

SD 

Simulation  is  a  useful  tool. 

6.00 

.00 

5.39 

.68 

Simulation  saves  time  in  the  aircraft. 

5.50 

1.00 

5.24 

.88 

Simulator  can’t  be  built  that  handles  like 
the  aircraft. 

5.25 

.96 

3.92 

1.00 

Time  saved  in  aircraft  from  simulation  is 
negligible. 

2.00 

.82 

2.92 

1.05 

Tasks  learned  in  a  simple  simulator  will 
transfer  to  the  aircraft 

5.00 

.00 

5.03 

.64 

1 .  A  high  rating  indicates  strong  agreement  with  the  item. 


Table  8 


Mean  Instructor  Pilot  Ratings  by  Item  Content  (6-pt.  Scale)1 


Description 

PST 

SFTS 

Mean 

SD 

Mean 

SD 

General  Attitude  Toward  the  Simul 

iator 

Some  things  students  learned  in  simulator 
hindered  performance  in  aircraft 

5.50 

.58 

4.75 

.50 

Simulator  Effectiveness  Ratings:  Basic  Instruments 


Straight  and  level  flight. 

5.50 

.58 

5.25 

.50 

Timed  turns. 

4.25 

2.22 

5.25 

.50 

Steep  turns. 

4.25 

2.22 

5.25 

.50 

Climbs  and  descents. 

4.25 

.96 

5.25 

.50 

Trim  control. 

1.00 

.00 

4.75 

.50 

Standard  rate  turns. 

3.75 

2.22 

5.25 

.50 

Climbing  turns. 

3.00 

1.63 

5.25 

.50 

Acceleration/  Deceleration. 

4.25 

.96 

5.25 

.50 

Descending  turns. 

3.00 

1.63 

5.25 

.50 

Simulated  engine  failure  at  altitude. 

5.25 

.50 

5.00 

.82 

Unusual  attitude  recovery. 

5.25 

.50 

5.25 

.50 

Simulator  Effectiveness  Ratings:  Advanced  Instruments 


Instrument  takeoff. 

5.50 

.58 

3.75 

1.89 

Missed  approaches. 

6.00 

.00 

5.00 

.82 

Holds. 

5.75 

.50 

5.00 

.82 

Instrument  approaches. 

6.00 

.00 

5.50 

.50 

1.  A  high  rating  indicates  strong  agreement  or  that  the  simulator  was  very  effective. 
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Table  9 

Instructor  Pilot  (IP)  vs.  Student  Pilot  (SP)  Ratings  of  Simulator  Effectiveness 
by  Item  Content,  for  All  Maneuvers  (6-pt.  Scale)1 


Description 

PST 

SFTS 

Mean 

Mean 

IPs 

SPs 

IPs 

SPs 

Some  things  learned  in  simulator 
hindered  performance  in  aircraft. 

5.50 

2.55 

4.75 

3.56 

Straight  and  level  flight. 

5.50 

5.40 

5.25 

4.67 

Timed  turns. 

4.25 

5.20 

5.25 

4.72 

Steep  turns. 

4.25 

5.00 

5.25 

4.83 

Climbs  and  descents. 

4.25 

5.05 

5.25 

4.67 

Trim  control. 

1.00 

2.45 

4.75 

3.61 

Standard  rate  turns. 

3.75 

5.00 

5.25 

4.89 

Climbing  turns. 

3.00 

4.80 

5.25 

4.89 

Acceleration/  Deceleration. 

4.25 

5.00 

5.25 

4.61 

Descending  turns. 

3.00 

4.85 

5.25 

4.83 

Simulated  engine  failure  at  altitude. 

5.25 

4.80 

5.00 

4.65 

Unusual  attitude  recovery. 

5.25 

5.30 

5.25 

5.17 

Instrument  takeoff 

5.50 

4.45 

3.75 

3.38 

Missed  approaches. 

6.00 

5.15 

5.00 

4.72 

Holds. 

5.75 

5.45 

5.00 

4.78 

Instrument  approaches. 

6.00 

5.55 

5.50 

4.72 

1 .  A  high  rating  indicates  strong  agreement  or  that  the  simulator  was  very  effective. 


Instructor  pilot  spontaneous  comments.  IPs  did  provide  some  comments  to  the  open- 
ended  question  at  the  end  of  the  questionnaire.  Of  the  four  IPs  who  completed  the  questionnaire, 
two  provided  this  input.  Table  10  presents  these  responses  by  category.  In  spite  of  the  small 
number  of  responses,  we  can  see  some  similarity  to  students’  perceived  strengths  and  weaknesses 
of  the  two  simulators.  The  Frasca  PST  was  seen  as  a  good  proof  of  the  concept  of  a  low-cost 
simulator,  but  one  in  need  of  improvement,  especially  with  regard  to  the  software  flight  model  and 
trim  control.  These  deficiencies  were  seen  as  having  diminished  its  effectiveness  in  the  study.  On 
the  other  hand,  IPs  perceived  the  2B24  SFTS  as  a  good  BI  trainer  whose  AI  usefulness  was 
hindered  by  a  lack  of  additional  navigation  radios  and  instruments,  and  the  general  dissimilarity  in 
configuration  of  its  cockpit  to  that  of  the  TH-67.  One  IP  mentioned  that  this  was  initially 
confusing  to  students,  who  eventually  were  able  to  readjust  to  the  TH-67’s  instrument  array. 
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Table  10 


Content  Categories  of  Spontaneous  Comments  of  Instructor  Pilots  on  Postexperimental 
Questionnaire 


Simulator 

Content  Category 

Number  of 
Mentions 

PST 

Flight  model  is  of  poor  quality. 

2 

Not  having  an  operating  trim  indicator  hindered  experiment. 

1 

RMI  and  HSI  should  be  duplicated  on  left  side  of  cockpit. 

1 

Visual  display  system  benefits  training. 

1 

Visuals,  terrain  database  need  to  be  more  accurate. 

i 

Having  cockpit  similar  to  TH-67  benefits  training. 

i 

Instructor  Operator  Station  needs  to  be  more  accessible. 

i 

SFTS 

Does  not  have  VHF  navigation  radios. 

2 

A  great  BI  trainer,  but  not  for  AI. 

1 

Cannot  teach  instrument  takeoff. 

1 

Students  initially  confused  by  differences  in  instruments,  but 
catch  up  to  PST  students  after  about  seven  flights. 

1 

Use  of  desktop  simulators.  Four  of  the  38  participants  reported  having  used  a  desktop 
flight  simulator  within  the  past  year.  This  reported  rate  of  usage  is  lower  than  expected  by  the 
investigators,  when  compared  with  the  results  of  a  study  by  Dunlap  and  Tarr  (1999),  who  found 
that  47%  of  U.S.  Navy  primary  flight  training  students  reported  having  used  PC-based  simulation 
software.  The  low  rate  of  self-reported  usage  in  the  present  study  precludes  any  meaningful 
comparisons  between  users  and  nonusers.  This  question  was  asked  because  USAAVNC  is 
interested  in  the  possible  benefits  of  pretraining  on  commercially  available  flight  simulation 
software. 

Supplemental  correlational  analyses.  In  addition  to  the  between-group  comparisons, 
correlational  analyses  were  run  on  the  time  to  proficiency  data,  with  BI  and  AI  put-up  and 
checkride  grades  as  the  criteria.  This  was  essentially  an  exploratory  data  analysis,  because 
no  prior  hypotheses  were  entertained  as  to  which,  if  any,  BI  and  AI  maneuver  tasks  would 
correlate  the  most  highly  (or  at  all)  with  the  criteria.  The  reader  should  also  note  that  the 
measurement  of  these  two  variables  was  quite  different;  for  the  maneuvers,  it  was  the  training 
IP’s  estimate  of  how  many  hours  were  required  for  the  student  pilot  to  reach  proficiency  on  a 
given  task;  for  the  checkrides,  it  was  the  overall  numerical  score  that  the  SIP  gave  the  student. 
One  possible  benefit  of  these  analyses  would  be  insight  into  which  maneuver  tasks  showed  the 
strongest  relationship  to  graded  checkride  performance.  It  should  be  recalled  that  checkrides  are 
administered  by  U.S.  Army  IPs,  whereas  the  daily  performance  evaluations  are  performed  by 
contractor  IPs;  hence,  the  two  are  independent  measures  of  student  performance.  Table  1 1 
presents  correlations  for  BI  maneuver  tasks  on  BI  and  AI  checkride  grades;  Table  12  presents 
the  same  for  AI  maneuver  tasks. 
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Table  1 1 


Pearson  Correlations  of  Times  to  Proficiency  on  Basic  Instruments  (BI)  Maneuver  Tasks 
with  BI  and  Advanced  Instruments  (AI)  Checkride  Scores  (N  =  38) 


Grades 

BI  Checkride 
Score 

AI  Checkride 
Score 

AI  checkride  score. 

.04 

1.00 

BI  put  up  score. 

,50b 

,43b 

AI  put  up  score. 

.20 

.30 

BI  average  time  to  proficiency  (grand  mean). 

-62b 

-,40b 

AI  average  time  to  proficiency  (grand  mean). 

-.18 

-.38a 

Maneuver  Tasks 


Acceleration/deceleration. 

-.23 

-.30 

Climbing/descending  turn. 

-.30 

-,38a 

Climbing/descending  turn,  emergency  panel. 

-.27 

-.24 

Climbs  and  descents. 

-,43b 

-.36a 

Climbs/descents,  emergency  panel. 

-.32a 

-.34a 

Simulated  engine  failure  at  altitude. 

-,38a 

-.31 

Standard  rate  turn. 

-.44b 

-.31 

Standard  rate  turn,  emergency  panel. 

-.09 

-.36a 

Steep  turns. 

-.01 

-,47b 

Straight  and  level  flight. 

-,52b 

-.23 

Straight  and  level  flight,  emergency  panel. 

,36a 

-.27 

Timed  turns. 

-,51b 

.12 

Unusual  attitude  recovery. 

-,39a 

-.10 

Unusual  attitude  recovery,  emergency  panel. 

-.26 

-,43b 

Note,  a  =  p  <  05;  b  =  £  <  .01 


An  examination  of  Table  1 1  reveals  that  BI  times  to  proficiency  correlated  significantly 
with  BI  checkride  scores  for  eight  of  the  14  tasks.  All  correlations  were  in  the  consistent 
direction  with  one  exception;  the  significant  positive  correlation  between  straight  and  level  flight 
(emergency  panel)  and  BI  checkride  grade  appears  to  be  somewhat  counter-intuitive,  in  the  sense 
that  the  longer  it  took  a  student  to  master  the  task,  the  higher  was  his  or  her  BI  grade.  This 
finding  could  be  due  to  the  fact  that  this  is  the  first  of  the  emergency  panel  maneuvers,  and  that  it 
simply  might  take  longer  to  master  the  first  time.  Other  correlations  were  in  the  expected 
direction,  indicating  that  the  sooner  a  student  pilot  demonstrated  proficiency  on  a  task,  the  higher 
he  or  she  was  likely  to  score  on  the  BI  checkride.  Note  that  six  of  the  BI  times  to  proficiency  for 
certain  maneuvers  also  correlated  significantly  with  scores  on  the  end  of  phase  AI  checkride.  It  is 
also  noteworthy  that  the  average  times  to  proficiency  (the  grand  mean)  of  all  these  maneuver 
tasks  were  highly  correlated  with  both  checkride  scores.  However,  the  same  (grand  mean) 
average  times  for  AI  maneuvers  only  correlated  with  scores  on  the  AI  checkride. 
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Also  of  interest  is  the  lack  of  a  correlation  between  BI  and  A1  checkride  scores;  in  fact, 
the  correlation  is  effectively  zero.  The  reason  for  this  lack  of  correlation  is  not  simple  to  explain. 
One  possibility  could  be  the  early  timing  of  the  BI  checkride,  which  evaluates  skills  that  have  only 
recently  been  mastered,  and  which  are  the  building  blocks  to  the  more  complex  AI  skills  to  be 
acquired  later.  Thus,  the  BI  skills  may  be  assessed  before  they  have  had  time  to  become  folly 
integrated.  Another,  more  mundane,  explanation  could  be  the  weather,  which  obviously  affects 
performance  in  the  aircraft  and  not  the  simulator.  Students  in  this  study  from  one  class  in 
particular,  after  scoring  high  on  their  BI  evaluations,  had  to  complete  their  AI  checkrides  under 
marginal  weather  conditions.  As  a  result,  students  with  BI  scores  in  the  90s  received  AI  scores  in 
the  80s,  which  were  adequate  to  pass,  but  nonetheless  disappointing.  SIPs  indicated  that  winds 
and  poor  weather  conditions  that  day,  adversely  affected  performance  of  all  students  who  took  a 
checkride. 

Table  12 

Pearson  Correlations  of  Times  to  Proficiency  on  Advanced  Instruments  (AI)  Maneuver  Tasks 
with  Basic  Instruments  (BI)  and  AI  Checkride  Scores  (N  =  38) 


Maneuver  Tasks 

BI  Checkride 

AI  Checkride 

Score 

Score 

En  route  navigation. 

.10 

-.23 

ILS  approach. 

.13 

-.18 

Instrument  takeoff. 

.16 

-,44b 

Localizer  approach. 

.05 

-.10 

Localizer  holding. 

-.23 

-.10 

Lost  communication  procedures. 

-.17 

-.17 

NDB  approach. 

-.10 

-.25 

NDB  holding. 

-.11 

-,45b 

Precision  approach. 

-.25 

-.10 

Radio  communication  procedures. 

.06 

-,44b 

VOR  holding. 

-.08 

-,43b 

VOR  missed  approach. 

-.03 

-.11 

Note,  b  =  p_<  .01 


Table  12  shows  quite  a  different  picture  for  the  AI  tasks.  None  of  the  12  AI  maneuvers 
correlated  significantly  with  the  BI  checkride  scores,  whereas  four  maneuvers  correlated 
significantly  with  the  AI  checkride  scores.  Even  so,  most  of  the  correlations,  for  both  BI  and  AI 
times  to  proficiency,  seem  internally  consistent  and  do  show  some  potential  of  utility  as 
performance  measures.  The  grand  mean  times  to  proficiency  for  both  BI  and  AI  maneuvers 
tended  to  correlate  strongly  and  consistently  with  their  respective  checkride  scores.  Although 
these  measures  may  have  utility  for  predicting  checkride  performance,  a  much  larger  sample  size 
than  the  current  one  would  be  required  to  explore  this  possibility.  Hence,  any  further  post  hoc 
correlational  or  regression  analyses  would  be  beyond  the  scope  of  this  report. 
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•  * 


Discussion 


Training  Effectiveness  of  the  PST 

The  hypothesis  that  instrument  skills  learned  in  the  PST  would  show  stronger  transfer  to 
the  aircraft  than  those  trained  in  the  SFTS,  was  not  confirmed.  Instead,  students  were  able  to 
complete  Instrument  Phase  successfully,  regardless  of  the  simulator  type.  Data  collected  as  part 
of  the  training  effectiveness  assessment  imply  that  the  P  ST,  with  modifications,  could  be 
successfully  employed  as  an  instrument  trainer  for  U.S.  Army  EERW  student  pilots.  No  student 
pilots  who  trained  in  the  PST  were  set  back  to  later  classes,  nor  were  any  eliminated  from  flight 
training.  The  results  of  the  study  do  not  show  that  the  PST  has  a  clear  advantage  over  the  SFTS 
as  to  training  outcomes.  This  could  be  attributed  to  at  least  three  factors:  First,  the  measures  of 
performance,  especially  the  traditional  flight  grades  and  checkride  scores,  may  have  been 
insensitive  to  any  differences  that  existed  between  the  two  simulators  (see  Dohme,  1995,  for  a 
discussion  of  the  limitations  of  flight  grades).  Secondly,  the  previously  mentioned  technical 
problems  with  the  Frasca  PST  may  have  countered  any  benefits  inherent  in  using  a  device  that 
was  more  similar  to  the  aircraft.  Third,  the  POI  used  for  this  research  project  was  designed  for 
the  SFTS,  a  non-visual  simulator;  no  maneuvers  were  attempted  in  the  PST  that  could  not  be 
performed  in  the  SFTS.  Obviously,  the  Army  would  not  find  these  problems  acceptable  in  a 
production  device,  and  would  require  remediation  before  such  a  device  could  be  fielded.  All  of 
these  things  considered,  it  would  still  seem  that  such  a  PC-based  training  device,  by  virtue  of  its 
greater  economy  of  operation  and  maintenance,  would  be  a  viable  candidate  for  replacing  the 
SFTS.  Furthermore,  since  the  PST  is  a  visual  simulator,  it  would  be  reasonable  to  suppose  that, 
were  the  Army  to  acquire  this  type  of  device,  it  would  find  training  applications  beyond  the 
training  of  instrument  flight  skills.  Effective  exploitation  of  contact  IERW  training  would  require 
upgrades  to  the  software  flight  model  and  perhaps  to  the  terrain  database  as  well. 

Assessment  by  Target  Audience 

An  important  part  of  the  evaluation  of  the  PST  was  the  collection  of  input  from  the 
students  who  trained  in  it,  and  the  IPs  who  would  be  conducting  the  training  in  this  type  of 
simulator.  These  respondents  comprise  a  sample  of  those  who  will  be  using  the  device  in  the 
future.  Students  were  generally  more  positive  in  their  evaluations  of  the  PST  than  the  SFTS 
(though  not  significantly  so,  for  most  items)  in  spite  of  the  perceived  handling  problems  of  PST. 
By  contrast,  IPs  were  more  ambivalent,  favoring  the  SFTS  for  BI,  but  acknowledging  that  the 
PST  had  advantages  for  AI  training.  There  was  consensus  among  IPs,  that  the  primary  deficiency 
of  the  P  ST  was  poor  trim  control.  This  high  degree  of  agreement  was  reflected  in  a  mean  rating 
of  1  on  a  6-point  scale,  with  a  standard  deviation  of  0.  Likewise,  student  pilots  assigned  to  the 
PST  gave  it  lower  ratings  on  trim  control  than  those  assigned  to  the  SFTS;  in  fact,  this  was  the 
one  questionnaire  item  on  which  they  gave  significantly  higher  ratings  to  the  SFTS.  This  leaves 
little  doubt  that  the  Frasca  PST  had  a  serious  trim  problem  that,  in  the  experience  of  both  IPs  and 
students,  diminished  its  training  effectiveness. 
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One  perceived  advantage  of  the  PST  was  the  similarity  in  the  location  of  the  basic  flight 
instruments  and  radios  to  those  in  the  TH-67,  obviating  a  relearning  of  instrument  scan  patterns 
after  returning  to  the  aircraft.  The  dissimilarity  in  scan  patterns,  and  the  necessity  of  relearning 
the  correct  ones,  was  mentioned  frequently  by  students  who  had  trained  in  the  SFTS.  Another 
perceived  advantage  for  the  PST  was  the  presence  of  a  visual  display  system.  During  an  IMC 
approach,  the  pilot  must  be  able  to  acquire  the  runway  visually  in  order  to  terminate  to  a  landing. 
This  can  only  be  done  in  a  simulator  with  a  visual  display  system.  Both  IPs  and  students  believed 
that  breaking  out  of  the  overcast,  to  see  the  runway  in  the  correct  location,  was  reinforcing  and 
tended  to  build  self-confidence.  This  finding  is  important  in  light  of  the  fact  that  historically, 
instrument  trainers  and  simulators  have  not  had  visual  display  systems,  because  of  a  lack  of  a 
perceived  need  for  them.  The  inclusion  of  a  visual  display  system  also  imparts  versatility  beyond 

instrument  training. 

Recall  that  the  most  common  complaints  concerning  the  PST  were  its  poor  trim  control 
and  its  anomalous  responses  to  pitch  and  power  changes.  The  ARI  research  team  was  unable  to 
resolve  all  the  software  issues  related  to  these  problems,  which  forced  the  IPs  to  work  around 
them  during  training.  Likewise,  the  calibration  of  the  attitude  indicator  and  turn  indicator  m  the 
PST  were  determined  to  be  nonstandard,  adding  to  the  challenge.  These  factors  made  it  difficult 
to  perform  the  standard  rate  turns,  climbs,  and  descents  that  must  be  learned  during  the  first  nine 
hours  of  Instrument  Phase.  The  IPs  understood  that  the  PST  was  a  prototype  proof-of-concept 
simulator  and  that  these  problems  would  have  to  be  resolved  in  a  production  simulator.  In  brief, 
it  seems  that  the  Frasca  342  PST  was  a  potentially  effective  training  device  marred  by  a  poor 
software  aerodynamic  model. 


Conclusions 

The  reader  should  not  lose  sight  of  the  fact  that  this  was  a  comparison  of  two  simulation 
technologies,  not  just  two  simulators.  In  this  sense,  it  was  shown  that  a  PC-based  simulation 
environment  can  train  Instrument  Phase  tasks  successfully.  One  important  implication  is  the 
functionality  of  a  future  generation  of  training  simulators,  in  terms  of  what  is  required  to  train 
IERW  tasks.  This  research  project  demonstrated  that  student  pilots  could  receive  the  full 
simulator  portion  of  Instrument  Phase  training,  in  a  PC-based,  non-motion  simulator  with  a  simple 
visual  display  system.  Such  a  simulator  would  also  be  more  economical  to  operate  and  maintain 
than  the  one  currently  employed.  Secondly,  it  provided  evaluative  input  from  the  student  pilots 
themselves,  as  to  its  perceived  effectiveness  for  training  various  IERW  flight  maneuvers.  The 
study  uncovered  technical  problems  with  the  prototype  training  device,  which,  though  they  likely 
detracted  from  its  effectiveness  as  an  instrument  simulator,  pinpointed  issues  that  must  be 
resolved  before  any  device  of  this  type  is  acquired  by  the  U.S.  Army.  Should  the  Army  decide  to 
acquire  a  PST-like  simulator  in  the  future,  it  appears  that  such  a  device  would  be  usable  for 
training  BI  and  AI  skills,  with  the  proviso  that  the  software  aerodynamic  model,  and 
instrumentation,  represent  the  TH-67,  both  physically  and  functionally.  In  the  current  state  of  the 
art  for  PC  technology,  this  challenge  can  easily  be  met. 

One  encouraging  finding  was  that,  regardless  of  the  simulator  in  which  students  trained, 
the  time  required  to  reach  proficiency  in  the  simulator  on  BI  tasks,  in  many  cases  correlated 
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significantly  with  the  scores  received  on  the  AI  checkride.  This  imparts  some  degree  of  validity  to 
the  employment  of  proficiency-based  criteria  instead  of  traditional  flight  grades  as  indices  of 
mastery.  It  also  lends  support  to  Dohme’s  (1995)  position  on  the  superiority  of  training  to 
standard  to  lock-step,  hours-based  training.  Future  investigation  of  the  sensitivity  and  validity  of 
proficiency-based  measures  of  performance  would  seem  a  worthwhile  undertaking,  in  light  of 
these  findings. 

The  investigators  found  one  aspect  of  the  research  to  be  very  difficult.  This  was  finding 
reliable  and  comparable  cost  figures  for  direct  comparison  of  the  PST  and  the  SFTS.  It  is  obvious 
fiora  its  complexity,  age,  and  maintenance  burden  that  the  SFTS  is  a  more  expensive  device  to 
operate.  Its  operation  requires  a  large,  climate-controlled  building,  and  hydraulics  that  generate 
heat,  consume  power,  and  require  frequent  maintenance.  The  PST,  on  the  other  hand,  consists  of 
four  Pentium  75  computers,  plus  one  486  DX  66.  A  more  current  version  would  have  more  up  to 
date  PC  equipment,  which  could  easily  be  expanded  and  upgraded.  This  could  include  a  more 
current  visual  display  system.  Thirty-two  PST-like  training  simulators  could  be  located  in  any 
institutional  building  with  sufficient  floor  space,  and  would  not  require  additional  climate  control 
equipment,  outside  of  that  needed  for  the  comfort  of  the  occupants.  Civilian  computer  specialists 
could  maintain  the  PSTs,  with  little  additional  training.  This  stands  in  contrast  to  the 
specialization  required  to  support  the  SFTS.  It  should  go  without  saying,  then,  that  cost 
advantages  could  be  realized  by  acquiring  PC-based  simulators,  with  no  detriment  in  training 
effectiveness.  Each  SFTS  also  requires  a  specialized  console  operator,  who  can  either  be  a 
contractor,  an  Army  civilian,  or  an  enlisted  soldier.  The  IP,  who  is  in  the  simulator  cab,  cannot 
access  the  controls  on  the  IOS,  due  to  his  or  her  physical  location.  By  contrast,  the  PST  has  a 
very  simple  IOS,  located  on  a  table  behind  the  cab,  consisting  of  one  PC,  with  an  interactive 
program  that  can  be  mastered  by  the  IP  and  the  student. 

One  final  difference  in  the  two  simulators  should  be  noted.  The  SFTS  is  a  dedicated 
instrument  simulator,  and,  without  an  upgrade,  cannot  support  Primary  Phase  flight  skills  training. 
By  contrast,  the  PST  used  in  this  study  was  equipped  with  a  simple,  forward-projection  visual 
display,  which  could  potentially  support  some  visual  flight  maneuvers.  The  PST  display 
technology  is  dated.  Recently,  ARI  has  integrated  a  current  generation  of  PC-based  displays 
using  rear-projection  technology  that  are  far  superior.  At  this  writing,  an  effective  EERW 
instrument  training  simulator  could  consist  of  the  following  components:  A  high-end  PC-based 
three-to-five-channel  imaging  system  (1024  x  768  pixels),  with  60  Hz  display  update  rate,  three  to 
five  2.5  m  rear-projection  visual  displays,  a  PC  or  minicomputer-based  host  computer,  a  fully- 
populated  IFR  cockpit  shell,  and  control-loading  system.  No  motion  system  would  be  needed; 
however,  a  simple  seat-shaker  may  be  desirable.  A  rough  unit  cost  estimate  for  such  a  simulator 
would  be  approximately  $1,000,000. 

An  added  benefit  of  such  a  system  would  be  its  potential  for  training  a  much  broader  range 
of  flight  maneuver  tasks  than  the  SFTS.  Exactly  what  tasks  remains  an  empirical  question.  A 
future  research  project,  employing  such  an  updated  simulator,  could  determine  the  range  of  flight 
skills,  both  Primary  and  Instrument  Phase,  which  could  be  trained. 
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Appendix  A 

Student  Proficiency  Records 


ARI  IERW  Research  Project:  Simulator  Proficiency  Record:  Basic  Instruments 

SP  Name 

Rank 

SSN 

Instructor 

Date 

Training  Device  (Check  One) 

Frasca  PST 

2B24  SFTS 

Hours  to  Proficiency 

Please  indicate  for  each  task  the  Hours  at  which  the  SP  first  demonstrate! 
not  show  proficiency  on  the  task  during  the  simulator  training,  please  ind 

d  proficiency.  IfSPdid 
icate  ’’Not  demonstrated" 

Basic  Instruments  Tasks 

Full  Pane! 

Emergency  Panel 

Straight/Level  Flight 

Standard  Rate  Turns 

Compass  Turns 

Timed  Turns 

Climbing/Descending  Turns 

Steep  Turns 

Accelerate/  Decelerate 

SEF  at  Altitude 

Climbs/Descents 

Unusual  Attitude  Recovery 

Trim  Control 

Comments: 

A-l 


Appendix  A  (Continued) 


ARJ IERW  Research  Project:  Aircraft  Proficiency  Record:  Advanced  Instruments 


SP  Name 

Rank 

SSN 

Instructor 

Date 

Training  Device  (Check  One) 

Frasca  PST 

2B24  SFTS 

Hours  to  Proficiency  (i.e.,  Standard)  in  the  TH-67  Aircraft 

Please  indicate  for  each  task  the  Hours  at  which  the  SP  first  demonstrated  proficiency  in  the 
aircraft,  during  the  I A  Phase  of  IERW. 

Advanced  Instrument  Tasks 

Hours  to  Proficiency 

Comments 

Instrument  Takeoff 

Radio  Communication 

Missed  Approach 

Radio  Navigation 

Lost  Communication 

Holding: 

NDB 

VOR 

LOC 

Instrument  Approach: 

ADF 

PAR 

ILS 

LOC 

Emergency  Procedures 

Comments: 

A- 2 


Appendix  B 


TH-67  Frasca  Primary  Skills  Trainer  (PST)  Questionnaire  (SFTS  questionnaire  was  identical 
except  for  simulator  named) 

Administer  after  completion  of  Training 

Name  "Last  Four" _ 


We  would  like  to  ask  you  some  questions  about  your  perceptions  regarding  simulation  in  general, 
and  your  experience  in  the  Frasca  TH-67  Primary  Skills  Trainer  in  particular.  We  are  asking  for 
your  name  and  the  last  four  digits  of  your  SSN  for  data  analysis  purposes  only.  Your  answers 
to  the  questions  will  reported  as  aggregated  data  along  with  the  averages  of  other  people  who 
completed  the  questionnaire.  Your  particular  responses  will  not  be  identifiable. 

On  the  questions  that  follow.  Please  indicate  your  impressions  by  placing  an  X  in  the  appropriate 
box  on  the  rating  scale.  We  appreciate  your  cooperation  in  completing  the  questionnaire. 

PART  I:  GENERAL.  The  following  questions  pertain  to  simulation  in  general,  and  your 
perceptions  of  its  role  in  rotary  wing  training. 

1.  All  in  all,  I  believe  that  simulation  is  an  effective  tool  for  initial  flight  training. 


Strongly 

Agree 

Agree 

Agree 

Somewhat 

Disagree 

Somewhat 

Disagree 

Strongly 

Disagree 

2.  Simulation  is  a  good  investment  in  that  it  saves  training  time  in  the  aircraft. 


Strongly 

Agree 

Agree 

Agree 

Somewhat 

Disagree 

Somewhat 

Disagree 

Strongly 

Disagree 

3.  In  spite  of  all  the  technology,  a  simulator  can't  be  built  that  handles  like  a  real  aircraft. 


Strongly 

Agree 

Agree 

Agree 

Somewhat 

Disagree 

Somewhat 

Disagree 

Strongly 

Disagree 

B-l 
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Appendix  B  (Continued) 

4.  Use  of  simulators  may  save  some  flight  training  hours,  but  this  is  generally  a 
negligible  amount. 


Strongly 

Agree 

Agree 

Agree 

Somewhat 

Disagree 

Somewhat 

Disagree 

Strongly 

Disagree 

5.  Skills  acquired,  even  in  a  simple  simulator,  should  transfer  to  the  aircraft. 


Strongly 

Agree 

Agree 

Agree 

Somewhat 

Disagree 

Somewhat 

Disagree 

Strongly 

Disagree 

PART  II:  EVALUATION  OF  THE  SIMULATION  EXERCISE 

We  are  interested  in  your  perceptions  of  the  effectiveness  of  the  TH-67  Primary  Skills 
Trainer  (PST)  in  the  context  of  the  training  exercise,  which  you  just  completed.  Your  responses 
to  the  following  questions  would  be  of  great  value  to  us. 

6.  I  believe  that  that  the  out-the-window  view  of  the  PST  was  adequate  for  Basic  Instrument 
training.  (IF  THE  VISUAL  DISPLAY  WAS  NOT  USED,  INDICATE  "NOT  APPLICABLE") 


Strongly 

Agree 

Agree 

Agree 

Somewhat 

Disagree 

Somewhat 

Disagree 

Strongly 

Disagree 

Not 

Applicable 

7.  I  believe  that  some  of  the  techniques  that  I  had  to  learn  to  fly  the  PST  hindered  my 
performance  in  the  TH-67  aircraft. 


Strongly 

Agree 

Agree 

— 

Agree 

Somewhat 

Disagree 

Somewhat 

Disagree 

Strongly 

Disagree 

PART  III:  Basic  Instruments 

The  following  questions  concern  the  extent  to  which  you  believe  that  Basic  Instruments  training  in 
the  Frasca  PST  affected  your  performance  in  the  TH-67  aircraft.  For  those  question  items  below, 
please  indicate  the  degree  to  which  you  think  the  PST  was/was  not  helpful  in  doing  this,  for  those 
maneuvers  listed  below. 
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Appendix  B  (Continued) 
8.  Straight/Level  Flight 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

9.  Timed  Turns 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

1 

10.  Steep  Turns 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

1 1 .  Climbs/Descents 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

12.  Trim  Control 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

13.  Standard  Rate  Turns 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

14.  Climbing  Turns 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 
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Appendix  B  (Continued) 

15.  Acceleration/Deceleration 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

16.  Compass  Turns 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

1 

17.  Descending  Turns 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

! 

1 8 .  Simulated  Engine  F ailure/ Altitude 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

19.  Unusual  Attitude  Recovery 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

PART  IV:  Advanced  Instruments 
20.  Instrument  Takeoff 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

>  to  - 


Appendix  B  (Continued) 
21 .  Missed  Approach 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

22.  Holding:  NDB,  VOR,  LOC 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

23 .  Instrument  Approach 


Very  Helpful 

Helpful 

Somewhat 

Helpful 

Somewhat  a 
Hindrance 

A  Hindrance 

Very  Much  a 
Hindrance 

COMMENTS  (OPTIONAL). 
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Appendix  C 

Student  Background  Questionnaire 

U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences,  (ARI)  Fort  Rucker  Field 
Unit 

Use  of  Commercial  Desktop  Flight  Simulator  and  Aviation-Related  Games 

We  are  interested  in  determining  the  extent  to  which  IERW  student  pilots  have  had  experience  in  the  use 
of  PC-based  flight  simulation  and  aviation  video  games.  While  traditionally  thought  of  as  recreational,  these 
programs  may  be  potentially  valuable  teaching  tools.  This  information  will  be  used  strictly  for  research  purposes 
by  ARI. 

PART  I:  General  Background  Questions 

Name _ __  IERW  Class _  Today’s  Date _ 

Last  Four  Numbers  from  SSN _ Race _  DOB - 

Gender _ M  _ F  Rank _  AFAST  SCORE _ (estimate,  if  unknown) 

Prior  to  IERW,  how  many  flight  hours  have  you  had? 

_ None 

_ Hr  fixed  wing 

_ Hr  rotary  wing 

PART  II:  Use  of  Commerciallv-available  desktop  flight  simulations  (such  as  Microsoft  Flight  Simulator,  Strike 
Eagle,  Falcon  and  RAH -66  Comanche). 

In  the  past  year,  I  have  run  a  desktop  flight  simulator  or  aviation-related  game  on  a  PC. 


Yes  _ No 


IF  YES,  Please  indicate  the  name(s)  of  the  program(s)  and  the  approximate  number  of  hours  used. 


Name  of  Program 

Approximate  hours  per  month 

C-l 


